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Abstract 

Machine learning relies on the assumption that unseen test instances of a classification 
problem follow the same distribution as observed training data. However, this principle 
can break down when machine learning is used to make important decisions about the 
welfare (employment, education, health) of strategic individuals. Knowing information 
about the classifier, such individuals may manipulate their attributes in order to obtain a 
better classification outcome. As a result of this behavior—often referred to as gaming —the 
performance of the classifier may deteriorate sharply. Indeed, gaming is a well-known 
obstacle for using machine learning methods in practice; in financial policy-making, the 
problem is widely known as Goodhart's law. In this paper, we formalize the problem, and 
pursue algorithms for learning classifiers that are robust to gaming. 

We model classification as a sequential game between a player named “Jury" and a 
player named "Contestant." Jury designs a classifier, and Contestant receives an input to 
the classifier drawn from a distribution. Before being classified. Contestant may change 
his input based on Jury's classifier. However, Contestant incurs a cost for these changes 
according to a cost function. Jury's goal is to achieve high classification accuracy with respect 
to Contestant's original input and some underlying target classification function, assuming 
Contestant plays best response. Contestant's goal is to achieve a favorable classification 
outcome while taking into account the cost of achieving it. 

For a natural class of separable cost functions, and certain generalizations, we obtain 
computationally efficient learning algorithms which are near optimal, achieving a clas¬ 
sification error that is arbitrarily close to the theoretical minimum. Surprisingly, our 
algorithms are efficient even on concept classes that are computationally hard to learn. For 
general cost functions, designing an approximately optimal strategy-proof classifier, for 
inverse-polynomial approximation, is NP-hard. 



1 Introduction 


Studies have found that a student's success at school is highly correlated with the number 
of books in the parents’ household [IEKST10l| . Therefore, in theory, this attribute should he of 
great value when using machine-learning techniques for student admission. However, this 
statistical pattern is obviously open to manipulation: books are relatively cheap and, knowing 
that their number matters, parents can easily buy an attic full of unread books in preparation 
for admission decisions. 

This behavior is often called gaming: the strategic use of methods that, while not dishonest 
or against the rules, give the individual an unintended advantagej^The problem of gaming is 
well known and can be seen as a consequence of a classical principle in financial policy making 
known as Goodhart’s law: 

“If a measure becomes the public’s goal, it is no longer a good measure.” 

Goodhart's law is highly relevant for the practice of machine learning today. Machine learning 
relies on the idea that patterns observed in training data translate to accurate predictions about 
unseen instances of a classification problem. Machine learning is increasingly used to make 
decisions about individuals in areas such as employment, health, education and commerce. In 
each such application, an individual may try to achieve a more favorable classification outcome 
with little effort by exploiting information that may be available about the classifier. Goodhart's 
law suggests that if a classifier is exposed to public scrutiny, its prediction accuracy vanishes and 
it becomes useless. Indeed, concerns of gaming and manipulation are often used as a reason for 
keeping classification mechanisms secret, which is a major concern in credit scoring (cf. |GP14|| ). 
Secrecy is not a robust solution to the problem; information about a classifier may leak, and it is 
often possible for an outsider to learn such information from classification outcomes. Moreover, 
transparency is highly desirable and sometimes even mandated by regulation in applications 
of public interest. 

Our goal in this work is to formalize gaming in classification and to develop approaches and 
techniques for designing classifiers that are near optimal in the presence of public scrutiny and 
gaming. The hope is that this analysis may lead, in certain cases, to classifiers with performance 
comparable to ones that rely on secrecy. In other cases, our analysis may lead to the realization 
that secrecy is necessary for a good classification performance. 

As gaming entails strategic behavior, any attempt to formalize it must incorporate the 
strategic response of an individual to a classifier. We propose a general model for strategic 
classification, based on a sequential two-player game between a party that wishes to learn a 
classifier and a party that is being classified. This is different from the standard supervised- 
learning setup, which is commonly viewed as a one-shot learning process, in which an algorithm 
produces a classiher from labeled training examples. Our model combines the statistical 
elements of learning theory—namely, seeking a small generalization error given a finite number 
of training data—with a game-theoretic notion of equilibrium. This combination allows us 
to build classifiers that achieve high classification accuracy at equilibrium, when both parties 
respond strategically to each other. 

Informal description of our model and results. We model learning and classification as a 
sequential two-player game. The first player, named "Jury," has a learning task: she is given 

^See, for instance, http : / /www .thefreedlctionary. com/gamesmanship 
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labeled examples from some true classifier h, and must publish a classifier /. The second 
player, named "Contestant," receives an input to the classifier, and is given a chance to "game" 
it. That is. Contestant may change his input based on /. However, Contestant incurs a cost 
for these changes according to a cost function known to both players. Jury's goal is to achieve 
high classification accuracy with respect to Contestant's original input and the true classifier 
h. Contestant's goal is to be accepted by Jury, without paying too much to change his input. 
The cost function plays an important role in our framework as it determines the flexibility of 
Contestant in changing his input. Ideally, the cost function should capture ground truth or our 
best approximation thereof. 

Our contributions are the following: 

- For certain cost functions, we give an efficient strategy for Jury which approaches the 
optimal payoff. Surprisingly, this result holds even for concept classes which are com¬ 
putationally intractable to learn. The intuitive reason is that Contestant's changes to his 
input "smooth out" any intractability. 

- Those cost functions for which Jury has near-optimal algorithms include separable cost 
functions. This is a natural class of cost functions which generalize our introductory 
example of school admissions and books. We also obtain results for a broad generalization 
of these separable functions. 

- In contrast, we show that, for general cost functions—even for cost functions which are 
metrics, another nice class—it is hard to approximate the optimum classification score 
with reasonable accuracy. 

- We observe through experiments on real data that our approach leads to higher classifica¬ 
tion accuracy compared with standard classifiers in situations where even a small amount 
of gaming occurs. We also experimentally demonstrate the robustness of our framework 
to inaccuracies in our modeling assumptions and the modeling of the cost function. 

1.1 Our model 

We first describe an idealized version of the game, where Jury has perfect information. This 
will serve as a reference point for how well Jury may hope to do. We will later relax this to a 
version where Jury knows neither h nor V, and only sees labeled examples. 

Definition 1.1 (Full information game). The players are Jury and Contestant. Fix a population 
X, and a probability distribution V over X. Fix a cost function c : X x X —> 1R+ and a target 
classifier h: X ^ {-1,1}. 

1. Jury (who knows the cost function c, the distribution V, and the true classifier h) publishes 
a classifier / : X ^ {-1,1}. 

2. Contestant (who knows c, and /), produces a function A : X ^ X. 

The payoff to Jury is Pr;(..^p{h(x) =/(A(x))}. The payoff to Contestant is 
^x~v [/(AW)-c(x,A(x))]. 

Definition 1 1.1 1 is an example of a Stackelberg competition, which means that the first player 
(Jury) has the ability to commit to her strategy (a classifier /) before the second player (Con¬ 
testant) responds. We wish to find a Stackelberg equilibrium, that is, a highest-payoff strategy 
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for Jury, assuming best response of Contestant; equivalently, a perfect equilibrium in the 
corresponding strategic-form game. 

Notice that designing the optimum /, given h, V and c, for a finite X, is a conventional 
combinatorial optimization problem. We seek to label the points in X with +1 so that the 
expectation, over V, of h{x) ■ f{A{x)) is maximized. Here, A(x) is a best move of Contestant, that 
is, 

A(x) = argmax^g^/(j;)-c(x,j;). (1) 

We note that A(x) may not be well-defined, if there are multiple y which attain the maximum. 
In the following, we assume that Contestant may move to any of them; for simplicity, we do 
assume that if one of the maximum-attaining y is x itself, then A(x) = x. That is, if Contestant is 
indifferent between moving and not moving, he will default to not moving. We refer to the best 
payoff for Jury in the above full-information game at the "strategic maximum" of the game: 

Definition 1.2 (Strategic Maximum). The strategic maximum in the full-information game is 
defined as 

OPT;,(D,c)= max Pr [/i(x) =/(A(x))], 
where A(x) is defined as in ([^. Notice that A(x) depends on /. 

Remark 1.3. For intuition, notice that if c{x,x) = 0 (that is, it costs nothing for Contestant to stay 
where he is), then A(x) has the following characterization: 

- if f(x) = 1, then A(x) = x; 

- ^ff(x) = -H let y = argmin^g^. y(j,)=ic(x,y); then 

lx c(x,y)^2. 

Indeed, since Contestant is best-responding, he only makes a move from input x to point y if c{x,y) 
is strictly less than 2, which is the payoff he obtains by improving his outcome from “rejected" to 
“accepted." In this case, the quantity /(A(x)) in the definition of the strategic maximum becomes 

/(A(x))= max f{y). 

y:c(x,y)<2 

In Section]^ we show that, for general cost functions, the strategic maximum is NP-hard to 
approximate. However, we will also show that for a natural class of cost functions, it is possible 
to to design a classifier for which Jury's payoff is arbitrarily close to the strategic maximum, 
even when Jury has incomplete information. To formalize this, we introduce a second game, 
which we call the statistical classification game. In this game. Jury does not know the target 
classifier h for every point in X, but instead is given a few labeled examples from an unknown 
distribution V. Contestant best-responds to Jury's published classifier /. 

Definition 1.4 (Statistical Classification Game). The players are Jury and Contestant. Fix a 
population X and a probability distribution V over X. Fix a cost function c : X x X —> 1R+ and a 
target classifier h: X ^ {-IH}- 
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1. Jury (who knows only the cost function c) can request labeled examples of the form 
{x,h{x)), with X being drawn from V. She publishes a classifier f : X ^ {-1, 1}. 

2. Contestant (who knows c and f), produces a function A : X ^ X. 

The payoff to Jury is IPr^^-^x? {^(^) = The payoff to Contestant is lEx~v[f ^(^))]- 


1.2 Strategy-robust learning 

A learning algorithm in our setting has to accomplish two goals. First, it needs to learn the 
unknown target classifier from labeled examples. Second, it needs to achieve high payoff for 
Jury in the statistical classification game, by anticipating Contestant's best response. Below, 
we give two definitions of stategy-robust learning which combine these goals; the second is a 
stronger requirement than the first. In our first definition, we fix an unknown target classifier 
h, and demand an algorithm which, with high probability over the samples, returns a classifier 
/ guaranteeing a near-optimal payoff to Jury in the statistical classification game. In our second 
definition, we present a uniform notion: the learning algorithm must, with high probability, 
return a classifier that is guaranteed to work on any target classifier h in some concept class H. 

Definition 1.5 (Strategy-robust learning). Let C be a class of cost functions. We say that an 
algorithm ^ is a strategy-robust learning algorithm for C if the condition that follows holds. 
For all distributions V, for all classifiers h, all c € C and for all e and 3, given a description 
of c and access to labeled examples of the form {x,h{x)), where x ~V, A produces a classifier 
/ : X ^ {-1,1} so that, with probability at least 1-3 over the samples. 


Vr[h{x) = f{X{x))]>OPTAV,c) 

x^V 


( 2 ) 


where A(x) is defined as in (ID- 


One might expect, in line with PAC-learning [IVal84j| , that Definition 1.5 might restrict h to 
be in some concept class H. However, we will show that for a natural class C of cost functions, 
in fact it is possible to achieve strategy-robust learning with no dependence on h\ 

However, we may want to ask a bit more. Suppose that Jury builds a classifier for some 
property, and later wants to re-use the data to build a classifier for a slightly different property. 
For example, returning to the scenario from the introduction, suppose that the school admis¬ 
sions board collects data on students and tries to predict academic success. Later, the board is 
charged with recruiting to maximize the quality of the basketball team; they would like to use 
the same dataset to predict who will be a good student-athlete. Later still, suppose that the this 
data set is made public, and many other schools try to use it to predict many things. If enough 
different classifiers are trained on this data, the guarantee of Definition [L^ starts to degrade. A 
strategy-robust learning algorithm should succeed with high probability on a single classifier, 
but there are no guarantees (beyond what the union bound gives) if it is used repeatedly. This 
situation motivates the following definition. 


Definition 1.6 (Uniform strategy-robust learning). Let be a concept class and C be a class of 
cost functions. We say that an algorithm ^ is a uniform strategy-robust learning algorithm for 
(?f,C) if the condition that follows holds. For all distributions V, for all c € C and for all e and 3, 
with probability at least \ -3 over draws x ~ D, the following holds simultaneously for all h eH. 
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Given a description of c and access to labels {x,h{x)), A produces a classifier f : X ^ {-1, 1} so 
that 

Pr [h{x) = /(A(x))] ^ OPThiV, c) - e, (3) 

x^V 

where A(x) is defined as in 0. 

We will typically specify the number of labeled examples that the algorithm requires as a 
function of e, 6 and a parameter that depends on the domain size (e.g., the number of features). 


1.3 Our contributions 


Our main result is a strategy-robust learning algorithm, which comes with both uniform and 
non-uniform guarantees. Our algorithm is computationally efficient when the cost function 
comes from a broad class of functions that we call separable. In the non-uniform case, the target 
classifier h can be anything. In the uniform case, the algorithm is efficient as long as the concept 
class H is statistically learnable, but it notably does not require that H be efficiently learnable. 

Separable cost functions are functions of the form c{x,y) = max{0,C2(y) - Ci(x)}, where 
and C 2 are arbitrary functions, mapping the domain X into the real numbers. We take the 
maximum with 0 to obtain a nonnegative cost function. We will later see and discuss a number 
of natural examples of separable cost functions. 

Our main theorem, and our stronger result, is about uniform strategy-robust learning. 

Theorem 1.7 (Informal). Let Hbea concept class that is learnable from m examples up to error e and 
confidence 1-6, and let S be the class of separable cost functions. Then, there is a uniform strategy- 
robust learning algorithm for {TL,S) with running time and sample complexity poly(m, l/£,log(l/6)). 

In fact, (the formal statement of) this theorem implies a non-uniform result: 

Theorem 1.8 (Informal). Let S be the class of separable cost functions. There is a non-uniform 
strategy-robust learning algorithm for S with polynomial running time and sample complexity. 

Our main theorem (and the non-uniform corollary) can be extended to a more general class 
of cost functions, which are obtained by taking the minimum of k separable cost functions. We 
state only the uniform version here, the non-uniform version follows similarly. 


Theorem 1.9 (Informal). Let Hbea concept class that is learnable from m examples up to error e and 
confidence 1-6, and let be the class of minima ofk separable cost functions. Then, there is a uni¬ 
form strategy-robust learning algorithm for {H,S^^'^) with sample complexity po\y(m,k, 1/c, log(l/6)) 
and running time poly(m,exp(/c), l/£,log(l/6)). 


Theorem 1.9 applies to a broad class of cost functions: it is not hard to see that any cost 
function on a finite domain X can be written as a minimum of separable cost functions. Of 
course, the sample complexity in Theorem |1.9| depends on k, the number of cost functions 
involved. For general cost functions, k grows with |X| and might be quite large. However, many 
spaces admit a more efficient representation—for instance, if the cost function defines a metric 
that admits a small £-net, k depends only on the size of the net. Thus, A: is a parameter that 
interpolates nicely between tractable cases where k is small and the general case where k is 
unrestricted. 

The fact that the sample complexity in Theorem 1.9 might be large is unavoidable: for 
general cost functions, we have the following negative result. 
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Theorem 1.10 (Informal). There is a class of metrics S such that, unless P = NP, there is no efficient 
strategy-robust learning algorithm for S that achieves expected payoff within e = 1/|X|'? of the 
optimum, for any constant rj > 0. 

Recall that a distance function is a metric if it is non-negative, symmetric, and satisfies the 
triangle inequality. This result is an immediate corollary of the fact (which we will prove in 
Section]^ that approximating the strategic maximum for metrics is NP-complete. 

1.3.1 Experimental evaluation 

We experimentally evaluate our framework on real data from a Brazilian social network called 
Apontador. The data set deals with instances of review spam and was recently studied in the 
context of spam fighting |ICdCMBB14| . Classification of spammers is a natural setting for our 
methods, because spammers will of course try to game any automated attempt to identify them. 
We model a cost function that roughly reflects the loss in revenue that a spammer experiences 
when changing certain attributes. For instance, when a spam message contains a URL pointing 
to malware, it is costly for the spammer to remove this URL from his message as his message 
loses its intended purpose. Acknowledging that the modeling of a cost function can never 
be perfectly realistic, we evaluate our approach while explicitly taking into account several 
types of modeling inaccuracies. Specifically, we only assume that our cost function is roughly 
correct and that the amount of gaming is possibly below or above the threshold predicted by 
our theoretical framework. Our empirical observations demonstrate that even in the presence 
of significant modeling errors and only a small amount of gaming, our algorithm already 
outperforms a standard SVM classifier. Complementing our robustness analysis, we explore an 
approach for creating hybrid classifiers that interpolate between our classifier and standard 
classifiers that aren't by themselves strategy-robust. We observe that such hybrids often achieve 
an excellent trade-off between resilience to gaming and classification accuracy. 


1.4 Related work 


The deterioration of prediction accuracy due to unforeseen events is often described as concept 
drift and arises in a number of contexts. A sequence of works on adversarial learning is motivated 
by the question of learning in the presence of an adversary that tampers with the examples 
of a learning algorithm. Typical application examples in this line of work include intrusion 
detection and spam fighting. Early works considered zero-sum games PDDM'*'04| which are 
not very applicable to our problem as there are almost always cases where the payoff should 
be high for both players (e.g, a good student being admitted to a good college). More recent 
work considers alternative game-theoretic notions |BS09llBSllllBKST^IGSBS13j . The most 
closely related is the work by Bruckner and Scheffer |IBS1 If . which considered a Stackelberg 
competition for adversarial learning. A notable difference with our setup is that they define 
the equilibrium with respect to the sample, while we define it with respect to the underlying 
distribution. Our definition requires us to provide generalization bounds. Beyond this differ¬ 
ence, Bruckner and Scheffer focus on learning centered linear classifiers when the Euclidean 
squared norm is the cost function. The Euclidean norm is not separable and so our results are 
incomparable. Stackelberg competitions have also been studied extensively in the context of 
security games IKYK+111[KCFT0| . 
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2 Separable cost functions 


We begin by studying the class of separable cost functions, which arise naturally in the context 
of gaming. To motivate the definition, recall the example of the school board which wants to 
exploit the correlation between parents' books and students' performance. In this (admittedly 
rather stylized) example, the cost to Contestant from moving from a household x e X with 50 
books to a household y eX with 100 books is simply the cost of the the additional books. 

More generally, this logic applies to any situation where Contestant can assign a cost to each 
state xeX, independently of how it was reached. If the cost of a state x is g(x), then the cost 
to Contestant of moving from x to p is simply any additional cost: c{x,y) = max{0,^(p) -^(x)}. 
For example, suppose that Jury is designing a spam filter, and Contestant wishes to send an 
email. Independently of the spam filter. Contestant wants his message to serve a purpose such 
as advertising or distributing malware. We can assign a score ^(x) to each message in x € X that 
expresses how much utility the spammer experiences when this message is delivered without 
being classified as spam. For example, a message is significantly less useful for the spammer 
after the URL pointing to malware has been removed. The expression max{0,g(p) -g(x)} then 
captures the loss in utility (or expected revenue) when moving from x to y. We will return to 
this example in detail in our experimental evaluation in Section]^ 

With these examples in mind, we define a separable cost function as follows. 

Definition 2.1. A cost function c(x,p) is called separable if it can be written as 


c(x,p) = max{0,C2(p)-Ci(x)}, 


for functions Ci, C 2 : X ^ IR satsifying Cj (X) c C 2 (X). 

Above, the term "separable" is a slight abuse of terminology, because the cost function 
cannot be negative, and because of the assumption about Ci(X) c C 2 (X); a truly "separable" 
function would be of the form C 2 (y)-Ci(x), for arbitrary Ci,C 2 . However, we will stick with it for 
simplicity of exposition. The two extra conditions are natural for cost functions. The maximum 
with 0 ensures that the cost function is non-negative. The condition Cj(X) c C 2 (X) means that 
there is always a 0-cost option (that is. Contestant can opt not to game, and can pay nothing). 

Another important special case of a separable cost functions are linear cost functions of the 
form 

c{x,y) = <a,(y-x)>^, 

for a € ]R”. With this cost function, each attribute can be increased independently at some 
linear cost, and can be decreased for free. For our arguments that follow, a linear cost function 
is helpful for intuition. 

Our main result is that for separable cost functions, there is a nearly optimal algorithm for 
Jury, with a uniform guarantee. The sample complexity and running time of this algorithm 
depend on the Rademacher complexity of the class Ti of classifiers. 

Definition 2.2. For a class T of functions / : X ^ IR, the Rademacher complexity of T with 
sample size m is defined as 


Rm(J^) ■= . x„,~V^ai . 0 


sup 


/ ni 

I i=l 


feT 


where are i.i.d. Rademacher random variables. 
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Our algorithm, given below as Algorithm has the following uniform guarantee. 

Theorem 2.3. Suppose the cost function c is separable, i.e., c{x,y) = maxjO, C 2 (y) - Ci(x)} and 
Ci(X) c C 2 {X). Let H be a concept class, and let V be a distribution. Let m denote the number of 
samples in Algorithm^ and suppose 


RM + 2 


V 


ln(nj+l) 


+ 




\n(2/6) ^ £ 
" 8 


8 m 


Under these conditions, -with probability at least 1 - 6, ([^ holds for all h eH. 

Notice that Theorem |2.3| indeed implies the "informal" version, Theorem |1.7[ That is, if H 
is statistically learnable (i.e., Rm{Td) decays inversely polynomially with m for all distributions 
L?, or sufficiently that the VC dimension of H is boundec^l, then Algorithmic is a efficient, 
uniform strategy-robust learning algorithm for H. 

It is worth pointing out that Algorithm is computationally efficient as long as H has 
low sample complexity—even if H itself is not computationally efficiently learnable! As 
we mentioned above, the proof of Theorem 2.3| also implies that our algorithm satisfies the 
following non-uniform guarantee. 


Corollary 2.4. Suppose the cost function c is separable. Let m denote the number of samples in 
Algorithm^ and suppose that 

„ / ln(m+l) , l ln{2/6) ^ £ 

\ m ' y Sni g * 

Then with probability at least b, holds for all distributions V. In particular, Algorithm^is an 
efficient (non-uniform) strategy-robust learning algorithm. 


Corollary |2.4| follows from Theorem |2. 3 by setting H = [h], the singleton containing the 
fixed target classifier h. Indeed, in this case R„i{'H) - 0. 

Before proving Theorem 2.3 we state the algorithm and discuss the intuition behind it. In 
Figure[C we illustrate the idea for a linear cost function, c(x,y) = {a,y-x)_^. Because moving 
perpendicularly to a is free for Contestant, Jury may as well choose a classifier / that accepts 
some affine halfspace whose normal is equal to a (see Figure [C. Thus, the only issue is finding 
the correct shift for this halfspace. Because the calculated shift can only be based on samples, 
we choose the shift that is empirically the best. The latter can be calculated quickly because it 
is a one-dimensional problem. 

For a more general separable cost function 


c{x, y) = max{0, Ciiy) - Ci (x)}, 

by the same argument. Jury may as well return a classifier C 2 [t] of the form: 


C2[t](x) = 


1 if C2{x) > t 
-1 if C2(x) < t 


(xeX) 


for some t. Algorithmic gives the details, and we proceed with the proof below. 
^Indeed, if d is the VC dimension of H, we have 


Rminx 


2dlog(em/d) 


for all distributions V (notice that depends on V). 
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Figure 1: Suppose the optimal classifier for Jury is / (which accepts the dark gray region), and the cost 
function is c{x,y) = {oc,y- x)^. Because moving perpendicular to a is free for Contestant, then the payoff 
for Jury if she plays f' (shown above, which accepts the light gray region) is the same as her payoff if she 
plays /. Indeed, suppose that the agent x shown above would be willing to move to y to get accepted by 
/. Then x' would also be willing to move to y, because the cost is the same. Thus, Jury may restrict his 
or her search to classifiers f' that accept all points in some affine halfspace whose normal is equal to a. 


Algorithm 1: A\ gaming-robust classification algorithm for separable cost functions 

1 Inputs: Labeled examples (x^,/i(xi)),...,(x^,/i(x^)) from x, ~V i.i.d.. Also, a description 
of a separable cost function c{x,y) = maxjO, C 2 (y)- Ci (x)}. 

2 For z = let 


L :=Cl(X;) 


_ |max(c2(X) n [L, L + 2]) CjiX) n [f,, f,- + 2]^^ 
|oo c 2 (X)n[L,L + 2] = 0. 

For convenience, set s^+i = oo. 

3 Compute 

m 

^( 5 ,) := ^J^l[h(xj)^Ci[si-2]{xj)Y 

;=i 


4 Find i*, I ^i* ^m + l, that minimizes err(S;). 

5 Return: / := C2[S;.]. 
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Remark 2.5 (Input to Algorithm[^. Algorithmtakes a cost function c{x,y) = max{0,C2(p) - 
Ci(x)} as an input, and it returns some threshold function based on ci- We have heen a little 
sloppy about how exactly c should be represented. A quick inspection of the algorithm shows 
that in order to compute the threshold, A needs only black-box access to cj, and enough access 
to Cj to determine CjiX) fi [t,-, t, -I- 2]. In order to return the classifier /, A additionally needs 
whatever access to C 2 it is expected to return. For example, if we only ask that A be able to 
provide black-box access to /, then black-box access to C 2 suffices for this step. If we ask that A 
return a short description of /, then a short description of C 2 suffices for this step. 


Proof of Theorem 2.3 Assume for simplicity that the cost function satisfies c{x,y) ^ 2, for all 
x,y e X. First, for any mapping f : X ^ {-IA}, define 


F(/) := jx : max{/(y) : c{x,y) < 2} = 1 } 

= jx : {3y e X)(f{y) = 1, c{x,y) < 2)} 

= jx : ci(x) > min{c2(F) : fiv) = 1}" 2 } • 

Claim 2.6. F(/) is the set of x € X such that /(A(x)) = 1 when A is a best response of Contestant. 

Proof. Indeed, for x € F(/), there exists some y such that f(y) = 1, so that the payoff to Con¬ 
testant when he plays A(x) = y is equal to 1 - c{x,y) > -1. On the other hand, suppose that 
Contestant plays A(z) € X for some with f{z) ^ 1. Then the best payoff of Contestant is equal to 
-1 -c(x,z) < -1, because c(x,z) > 0. So, the best response of Contestant is to choose A(x) = y for 
some y with f{y) = 1. This establishes that F(/) c {x € X : /(A(x)) = 1}. 

For the other direction, suppose that /(A(x)) = 1. Then there is some y € X so that 


1 - c(x,y) > -1 


■ minc(x, z) = -1, 

zeX 


using from the definition of separability that Cj(X) c C 2 (X), and hence for all x. 


minc(x,z) = minmaxjO, C 2 (z) - Ci(x)} = 0. 

zeX zeX 

In particular, c(x,y) < 2, and so x € F(x). This establishes that 


{x€X:/(A(x)) = 1}cF(/), 


and proves the claim. 


Claim 2.6 is the only place in the proof where we need either of the extra conditions in 
Definition 2.1 (that c(x,y) > 0 and Ci(X) c C 2 (X)). 

Given this characterization of r(/), we next argue that we may replace / by a much more 
structured function /' so that F(/) = F(/'); in particular, the payoff to Jury under / will be the 
same as under /', and so we can restrict our attention to these more structured functions. For 
any /, let 

1 if C 2 (y) ^ min{c 2 (z) : f(z) = 1} 

-1 otherwise . 


f'iy) 


(4) 


Then we have 
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T(f) = {x : ci(x)>min{c 2 (y) : f(y) = l}-2} 

= {x : ci{x) > min{c 2 (y) : f'{y) = 1} - 2) 

=nn- 

In particular, for any true classifier h eH, the payoff to Jury if she plays / is the same as if she 
plays /': 

]P{h{x) = max{f{y) : c(x,j;) < 2}} = P|x € (r(/)A{y : h{y) = l]f} 

= lP{xe{T(f')A{y:h{y) = l})^] 

= ]P{h{x) = max{/'(y) : c{x,y) < 2}) . 

Above, A denotes symmetric difference. Thus, it suffices to consider classifiers of the form of 
That is, our classifier may as well be equal to C 2 [s], for some s € C 2 (X) U {oo), where s plays 
the role of min{c 2 (z) : /(z) = 1}, and s = oo means that there is no z such that /(z) = 1. Let 

S :=C 2 (X)U{co} 

be the set of these relevant values of s. Recall the definition of s, from Algorithm[^ For s € S, 
we hav41] 

r(c 2 [s]) = {x : Ci(x)>s-2} . 

The best possible payoff to Jury is obtained by finding the best threshold s, i.e., 

OPTi^{V,c) = 1 -inf{err(s) : s € S }, 

where err(s) := lP{h{x) Ci[s - 2](x)}. In Algorithm[^ Jury returns / = C 2 [S;.], and as above the 
payoff to Jury from this / is equal to 

P{/i(x) ^ Ci[Sj. -2](x)} = 1 -err(s;.). 

Thus, to prove Theorem |2.3[ it suffices to show that for all heH, 

err(s;.) < inf{err(s) : s € S } + £ . (5) 

To establish this, we first observe that there is no loss of generality in Algorithm [^by considering 
only the s,, z = 1,..., m + 1, where as in Algorithm[^we set s^+i = oo. 

Claim 2.7. 


err(S;.) = min{err(s,) : z = l,...,m + 1} 

= inf{eff(s) : s € S } . 

Proof. The first equality is just the definition of i*. The second equality follows from the fact 
that 

m 

;=1 

only changes when Cj[s - 2]{xj) changes for some j. Thus, by construction, this sum takes on 
every possible value (as s ranges over S = C 2 (X) n {oo}) at the points s,-, z = 1,..., zzz + 1. ■ 

^ As usual, oo - 2 = oo. 


12 





Claim 2.8. With probability at least 1 - S, for all heH and for all s e S, 


|err(5)-err(5)| < + 

In particular, under the conditions of Theorem \2.3\ with probability at least \-b, 

sup {|efr(s) - err(s) \ ■. h eH, s eS] < e/2 . 

Proof Writing out the definition of eff and err, we need to bound the absolute value of the 
difference 

m 

efr(s) -err(s) = ^ J^l[h{xj) ^Ci[s- 2](xy)} - [1 IHx) ^ci[s- 2](x)}] 

;=i 

simultaneously for all h e H, s e S. By standard arguments (see, for example. Theorem 3.2 
in |BBL05] ), for all h eC,s e S, 

|erf(s)-err(s)| < 2R,„(W) +, (6) 

where 

W = {h-ci[s-2] : hen,seS] . 

Thus, it suffices to control the Rademacher complexity of W, which is in turn controlled by 

R,n{X)^2{R„,{n) + RM)> ( 7 ) 


where y = {ci[s - 2] : s € S}. Note that, because all the functions in U T’ are +l-valued. 


h{x) • Ci[s - 2](x) = \ h{x) + ci[s - 2](x)| - 1 

for every x. Inequality ([^ follows from a contraction principle (see, e.g.. Theorem 4.2 in [ILT91IJ ) 
and the definition of the Rademacher complexity. 

It remains to bound R^^fy). Fix xi,...,x„, € X and sign flips cr; € {-1,1}. As in the proof of 
all of the values that YJiLi - 2](x,) takes on as s ranges over S are attained at 

€ X, using a Chernoff bound and the union bound, and 


2.7 


Claim 

{si,...,s„,+i}. Thus, for fixed xi,...,x 


integrating to bound the expectation, we obtain 


®cr[sup{i^^ff,Ci[s-2](X;) : S € S}] 
m 

= E^[sup{A^ff;Ci[s^ -2](x,) : ;■ = l,...,m + l}] 

! = 1 

< 2 

V m 

Thus, we have 

^m{y) < 

and altogether inequality ([^ implies that for allh eH and s € S, 

I err(s) - err(s) | < 4 |r,„(H) + 2^^^) + , 

which completes the proof of the claim. ■ 
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Claims 2.7 and 2.8 establish Theorem 2.3 Indeed, we have, with probability at least 1-6, 
for all h eC, 


err(s,'.) < efr(s,.) + e/2 

= inf {Sr(s) : s € S} + e/2 
< inf {err(s) : s € S} + £ , 

establishing inequality ID and completing the proof. ■ 


3 General cost functions 

While separable cost functions are quite reasonable, they do not capture everything. In this 
section, we consider more general cost functions. We extend Algorithm [^to work for a cost 
function that is the minimum of an arbitrary set of separable cost functions. This is a much 
broader class. In fact, every cost function can be represented as the minimum of separable cost 
functions, although not necessarily very parsimoniously. 

Proposition 3.1. Let X be any finite set and let c : X x X ^ IR be any mapping. Suppose 

D ^ max{c(x,y) : x,y eX]. 


Under these conditions, 

c{x,y) = min {c{w, z) + D ■ 1 {x iv} + D ■ 1 {y z} : iv,zeX} . 

Since each of the cost f unct ions = c(w,z) + D ■ l{x ^w} + D ■ l{y ^ z} is a. separable 

cost function. Proposition |3.1 [ implies that any c can be written as the minimum of |Xp cost 
functions. The sample complexity of our extension depends on the number of cost functions; 
since |X| may be quite large (possibly exponential in the parameter of interest). Proposition 
might not help. However, a smaller number of cost functions can be used if X has nice geometric 
structure. 

Proposition 3.2. Let X be any finite set and let c : X x X ^ IR be a metric. Let S be an e-net of 
X: that is, for every x e X, there is some s e S so that c{x,s) < e. Under these conditions, for every 
x,y e X, 

c{x, y) < min {c(x, w) + c{w, z) + c{z,y) : w, z € S} < c{x,y) + 4e . 

Thus, when c is a metric, our problem is very close to a problem where the cost function is 
the minimum of separable cost functions, and the number of cost functions we need to consider 
depends essentially on the covering number of the metric space (X, c). 

Algorithm [^is an adaptation of Algorithmfor cost functions of the form 

c{x,y) = min{b{x,y) : b e B}, 
where each function b eB is separable, i.e., 

b{x,y) = maxjO, 62 (p) “ • 
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Algorithm 2: A: gaming-robust classification algorithm for minima of separable cost 
functions _ 

1 Inputs: Labeled examples (xi,/i(xifrom x, ~'D i.i.d.. Also, a description 
of k separable cost functions h{x,y) = max{ 0 ,b 2 {y) -bi{x)} for b eB. 

2 For i = and b eB, set 


ti,b = bi{xi) 


max{b2(X) L,;,-h2]} 

oo 


if b2(X)n[tj}„ti^,, + 2]^(H 
if b2{X)r\[tii„ti^jj + 2] = (D 


and set = oo for all b eB. 

3 For each s e : i = 1), compute 


err(s):= ^ ^1 j/i(xy) min{Fi [sj, - 2 ](xy) :beB]j. 
;=i 


4 Find a s* that minimizes err(s). 

5 Return: /(x) = min{F 2 [sj](x) : b eB]. 


Theorem 3.3. Suppose the cost function c is the minimum of separable functions, 


c{x,y) = min{F(x,y) : b eB], 

where each b -.XxX ^Wtis separable. Let V be a distribution on X and suppose that Algorithm^ 
uses m samples, so that m satisfies 


Under these conditions, with probability at least 1-6, i) holds for all h eH and for the distribution 
V. The running time of Algorithm^is 0{m}^^). 


The intuition for Algorithm]^ is similar to that for Algorithmand is illustrated in Figure 
[^for the minimum of two linear cost functions. The proof of Theorem 3.3 is also similar to that 
of Theorem|2.3[ for completeness, we give it in Appendix [A| 


Remark 3.4 (Improvements for structured classes B). When the size of B is small. Theorem 3.3 


gives a nice bound. However, if B is large (as in our extreme example of the beginning of this section), 
these guarantees are not so good. An inspection of the proof (in Appendix^ shows that the term 

^ | 6 |ln(m+ ^ _R^( 7 f), where 


H = \ minFJsf, - 2 ] : 

heB 


s e 


(MX)uM) 


heB 


For some sets B of separable cost functions, this may be much smaller. 
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Figure 2: Suppose the the optimal classifier for Jury is / (which accepts the dark grey region), and the 
cost function is c{x,y) = min|(/l,y - x)^,{a,y - x)^|. For the same reasoning as in Figure 1 the classifier 
f has the same payoff to Jury as / does. Thus, Jury may restrict his/her search to classmers / that are 
the intersections of two affine halfspaces. 


4 NP-completeness 

What happens when the cost function c is not separable? It turns out that for general cost 
functions, any algorithm for Jury requires more than polynomial time to obtain a near-optimum 
classifier, unless P = NP. This holds true 

(a) even if the underlying distance function is a metric (another very natural class of cost 
function), and 

(b) even if the learning algorithm were given correct labels h(x) for all members x € X of the 
population, 

when the desired deviation e is inverse-polynomially small and the distribution V is uniform. 
The above statements are consequences of the following result: 

Theorem 4.1. Given a finite population X with the uniform distribution, a metric c on X, and a 
target labeling h\X\-^ {-l,-i-l}, it is NP-hard to compute the strategic optimum within e = for 
any constant rj > 0. 

Proof of Theorem \4.1\ We will reduce from 3Sat. Suppose we are given a 3Sat Boolean formula 
with n variables and m clauses Ci,...,C„,, where Q has three literal occurrences 

Lii,Lj 2 ,Li 3 - We now construct our instance of Strategic Optimum as follows. We need to 
specify X, h, and c. We begin by constructing a weighted population Y, which will consist of 
points y and positive integer weights w{y) for each y eY. Our population X will simply consist 
of w{y) identical copies of each y eY. Thus, |X| = ffyeY will also specify labels h{y) for 

each y eY, which the points x e X will inherit. Fix a number K (polynomial in m) to be chosen 
later. Our weighted population Y consists of: 

- 3m points for 1 < i < m and fc € {1,2,3}, corresponding to the literal occurrences in the 
clauses. These points each have weight w(Lii^) = K(m - 1 - ^) and label h{Lij^) = -1. 

- (^) points Pij for 1 < I <; < m corresponding to unordered pairs {Q, Cj} of clauses. These 
points each have weight w{Pij) - 2K and label hfPij) = -i-l. 
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- 9 • (^) points Qikj£, for 1 ^ i < j ^ m, and for k,£ e {1,2, 3} so that Ln^ is not the negation of 

Lj£. These points correspond to unordered pairs of literal occurrences of literal 

occurrences in different clauses which are not contradictory. They have weight w{Qiji£) = 1 
and label = -1. (Actually, their label does not matter). 

- One other point R with a huge weight w{R) = KM, for a very large value M, and label 
h{R) = -l. Choose M = 2Q. 

We next define a metric c : XxX ^ ]R+. It will take only two nonzero values, 1.5 and 2.5. 
Notice that this guarantees c satisfies the triangle inequality. We will choose c so that c(x, x) = 0 
and c(x,y) = c{y,x), and so c will indeed be a metric. To describe c, it suffices to describe the 
points which are "close," that is, which have distance 1.5. Further, it suffices to define c for 
points in Y, and we will extend it to X in a natural way: for points x,x' € X, if they come 
from the same y € Y, they will have distance 1.5; if x,x' come from y ^y' respectively, then 
c(x,x') = c{y,y'). The close pairs of points in Y are: 

- All pairs of the form [Pjj, Qiju}: 

- All pairs of the form {Pjj,R}; 

- All pairs of the form {Qijk£,Ltk} or {Qijk£,Ljf]. 

Claim 4.2. If the given formula is unsatisfiable, the number of points labeled +1 by the Jury’s 
optimum f is equal to 

which we call the baseline payoff. Otherwise, if the given formula is satisfiable, then there is a 
labeling f of the points with payoff at least b + K - 9(^). 

Proof. In the following, we will consider a graph with vertices Y. Two vertices x, y are neighbors 
in this graph if c(x,y) = 1.5. Let r(x) denote the neighbors of x in this graph. Thus, the best- 
response A to a classifier / is 


(x 


A(x) = 


X 


IF 


/(x) = 1 

/(x) = -1 and f(y) = -IVy € r(x), 
fix) = -1 and fiy) = l,ye r(x) 


where above if y in the last case is not uniquely defined Contestant can pick any such y. 

First observe that the baseline payoff is obtained by the classifier /(x) = -1 for all x € X, and 
so it is certainly acheivable. We now argue that Jury can do better if and only if the original 
formula was satisfiable. We make a few observations about Jury's optimal classifier /. 

- First, because of our choice of M, we must have f(Pij) = -1 for all i,]. Indeed, our choice 
implies that KM > |X| - KM; thus, if f{Pij) = 1 for some i,], then Contestant will set 
A{R) = Pij, and Jury will mis-classify the point R, and get a payoff worse than the baseline. 

- Next, f{Lik) = -1 for all i,k. Indeed, since h{Lik) = -1 and h{x) = -1 for all of the (Q-type) 
neighbors of Ljk, there can be no benefit to Jury for making /(L,fc) = -l-l. 
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- For each Pij, at most one Q-point Qikji in T(Pij) has /(Qikji) = +1- Indeed, each Q-point 
is connected to exactly one Pij, and once one of them is accepted by Jury, she can gain 
nothing by accepting additional points of r(P;y). 

Thus, the optimal / only assigns positive weights to Q points, and it does so to at most one 
Q-point in each r(P,y). Suppose that f{x) = -i-l for the set A of Q-points, and let B = rx^(A) be the 
set of L-points adjacent to A. Now, the size of B can vary based on how the literals overlap with 
the clauses. It satisfies 

^<|B|<2|A|, 

where the lower end is attained when there are complete collisions, and the upper end is 
attained when there are no collisions. Now consider the number of points of X that Jury 
classifies correctly under such an /. It is 

x(M + (3m-|B|)(m-l-^) + 2|A|) + j9j2j-|A|j = fo + 6, 


where 

6 = x(|B|(m-l-i) + 2|A|)-|A|. 

Consider this first term, which is multiplied by K. This is only positive when |B| = is as 
small as it can possibly be, which happens only if |A| = (^) and |B| = m. In this case, the first 
term is equal to K, and we have 8 = K-\A\^ K- 9{™). But this happens if and only if we can 
choose m different literals Ljj., one from each clause, so that no pair of them contradict each 
other; that is, if and only if the original formula was satisfiable. ■ 


Now the theorem follows quickly from the claim. We choose X to be a large polynomial in 
m, say for some small constant rj. Thus, |X| is on the order of Suppose there is 


a polynomial-time algorithm which approximates the strategic optimum up to e. Claim 4.2 
implies a contradiction for any 


K-9Q _ K-9Q 

K{3m + Q + M) + 9Q' 


Using our choice of K, for sufficiently large m the right hand side is at least |X| Thus, we 
have a contradiction whenever e < |X|“'i. ■ 


The metric constructed in the proof has "separability dimension" (the smallest number of 
separable functions needed to achieve it as a minimum) that grows linearly with the population. 
The same dimension appears in the exponent of the running time of the algorithm of the 
previous section. It is an interesting open problem to determine whether this exponential 
dependence is inherent; the other possibility is that the problem is fixed-parameter tractable with 
respect to the "separability dimension" parameter. We suspect that exponential dependence is 
necessary. 
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5 Experiments 

We conducted experiments on real data from a Brazilian social network called Apontador that 
provides location-based recommendations and reviews. The data set was introduced in the 
context of spam fighting in a recent work by Costa et al. [ICdCMBB14| and is available from 
the authors upon request. The data set consists of 7076 instances of so-called “tips" half of 
which are labeled as "spam". Tips are pieces of user-provided content associated with the 
places listed on Apontador. The paper distinguishes between different types of spam, but the 
distinction does not matter for us, so we will only consider one category. There are 60 features 
in total, but to facilitate the modeling of a cost function we restricted our attention to the 15 
most discriminative features as indicated by previous work pCdCMBB14] . We normalized all 
features of the data to have zero mean and unit standard deviation. 

The goal of our cost function is not primarily to capture monetary cost of changing certain 
attributes. Apart from attributes like "number of followers", most attributes are technically 
easy to change. Rather the goal of a cost function is to capture the loss in expected revenue that 
a spammer experiences when changing certain parts of the spam message. If, for instance, it is 
essential for the spam message to contain a URL or contact information, then the spammer ex¬ 
periences lost revenue when such information is omitted. Similarly, the spammer could choose 
to post his messages on the pages of lower-rated places, but such pages are less frequented and 
hence his utility decreases. Similar reasoning applies to the modeling of the other attributes. 
Cheap attributes are those that can be changed without a loss in utility for the spammer. For 
example, the "number of words" is not robust as the spammer can freely choose to write longer 
or shorter messages. 

With this intuition in mind, we model our cost function as a simple linear function truncated 
at 0 to make it non-negative. That is we consider a cost function of the form c{x,y) = {a,y -x)^. 
Truncation at 0 is a meaningful modeling decision, since a spammer doesn't derive any utility 
from, say, decreasing the number of his followers even though it is costly to increase this 
attribute. 

The cost vector a specifies for each attribute a coefficient quantifying the cost of changing 
that attribute. We do not attempt to construct as realistic a cost function as possible. We only 
distinguish between three types of cost: somewhat costly to increase (coefficient 1), somewhat 
costly to decrease (coefficient -1.0) and cheap to increase (coefficient 0.1). The concrete values 
of these coefficients are rather arbitrary and different choices may be more suitable. The next 
table details each feature with its description and its associated cost. For a more detailed 
explanation of these features, the reader is referred to [ICdCMBB14] . 
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Description 

Cost coefficient 

1 

Number of tips on the place 

-1 

2 

Place rating 

-1 

3 

Number of emails 

-1 

4 

Number of contact information 

-1 

5 

Number of URLs 

-1 

6 

Number of phone numbers 

-1 

7 

Number of numeric characters 

-1 

8 

SentiStrength score 

1 

9 

Combined-method 

1 

10 

Number of words 

0.1 

11 

Ratio of followers to followees 

1 

12 

Number of distinct 1-grams 

0.1 

13 

Number of tips posted by user 

0.1 

14 

Number of followers 

1 

15 

Number of capital letters 

0.1 


We made no attempt to arrive at a perfectly-realistic cost function. Instead our focus is 
on a qualitative comparison of our approach with a standard SVM classifier, which does not 
take gaming into account. We selected SVM as a representative classifier as it was shown in 
previous work [ICdCMBB14| to achieve high classification accuracy on this data set compared 
with other standard classifiers. For simplicity and increased interpretahility, we use a linear 
SVM which still achieves high accuracy. 

If we were to assume that our model of gaming and choice of cost function were perfectly 
correct, then a standard SVM would perform very poorly when compared with our algorithm. 
To obtain a more balanced comparison, we take modeling inaccuracies into account in our 
experiments. Specifically, we account for two potential inaccuracies in our model: 

1. The true cost function is not the one on which we train our algorithm. 

2. The amount of gaming varies and does not necessarily correspond to the threshold 
predicted by our theoretical framework. 

Finally, we explore a convenient way to interpolate between the classifier suggested by our 
approach and standard classifiers. This leads to different trade-offs which are more favorable 
in certain settings. 

5.1 Comparison with SVM under robustness to modeling errors 

We now show that our method is robust to significant modeling errors while simultaneously 
outperforming SVM even if only a small amount of gaming occurs. 

To formalize our error model, we assume that there is a true underlying cost function which 
differs from the cost function we feed into Algorithm We imagine that the true cost function 
is some mixture of the linear cost function described above, plus a squared Euclidean distance 
term: 

ctrue(vy) = (l-e){a,y-x)^ + e\\x-y\\^. (8) 

On the other hand, we run our algorithm on a cost function which is incorrect in two ways. 
First, it is separable, so it necessarily ignores the squared-distance term. Second, we do not 
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imagine that we have correctly identified a, and we replace it with some a'\ 

^assumed~ >y ~ ^)+ • 

The addition of the Euclidean norm in ([^ reflects the possibility that our separability assump¬ 
tion does not exactly hold. The difference between a and a' reflects the possibility that we 
may not even have accurately identified the separable part. We stress that not only does our 
algorithm not know the true cost function, it also does not know the parameter e, or how much 
a differs from a'. 

For our experiments, we considered a range of values of e, and we generated a' from a at 
random by adding Gaussian noise and re-normalizing. We develop our classifier using Cassumed^ 
but then for tests allow Contestant to best-respond to the classifier given the cost function Ctrue- 
We note that finding the best response to a linear classifier given the cost function Ctrue is a 
simple calculus problem. 

The other parameter we varied is the amount of gaming allowed. In our theoretical frame¬ 
work above, the Contestant is always willing to pay a cost of up to 2, since his payoff for 
switching is 1 - (-1) = 2. To relax this assumption and vary the amount of gaming allowed, we 
multiply both Ctrue and Cassumed by 2/t) we say that this allows t units of gaming. Notice that 
by the definition of C(.j.ug, this means that the Contestant is willing to move distance t in the 
direction of a, and possibly more in other directions. As mentioned above, we have normalized 
the standard deviation of all attributes to be 1. 

Within the above error model, we compare our algorithm with SVM as a representative 
standard classifier. Figures show that our algorithm outperforms SVM, even under a small 
amount of gaming, and even in the presence of significant modeling errors. 

5.2 A hybrid approach for higher accuracy 

In practice it is convenient to start from a standard classifier and make it more robust to gaming 
and as opposed to adopting an entirely new classifier. Our framework gives a convenient way 
to incorporate a set of known classifiers into the design of a strategy-robust classifier. As we 
show below this can lead to more favorable trade-offs between gaming and accuracy. 

The basic idea is to use each known classifier as a feature to which we assign a positive 
weight in the cost function. In other words, we stipulate that the classifier is by itself a 
somewhat reliable attribute of the data. Below we try out this hybrid approach by combining 
our classifier with the standard SVM classifier. Indeed, we find in our experiments that the 
hybrid has higher accuracy in a robust range of parameters. This is shown in Figure]^ 

In the case of a linear SVM, the decision boundary is given by a vector ji and we can simple 
add this vector to our cost function. We assume that the true cost function Ctrue is as above, but 
we modify as. 

Cassumed(V f) = {(I - y)a'+ y- x)^ , 

where fi are the SVM coefficients learned from the training data set. 
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Figure 3: Left: Our algorithm compared with SVM as the amount of gaming is increased. The x-axis 
tracks the amount of gaming, which is quantified as described above. The parameter e in Ct^ue is specified 
in the legend. We have set sin0(a,a') = 0.394 (again, a' was randomly generated from a by adding 
Gaussian noise and re-normalizing). Right: Our algorithm compared with SVM as the angle between a 
and a' increases. The x-axis measures the angle sin0(a,a'). The amount of gaming was fixed at 1.0, and 
the parameter e in is specified in the legend. 
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the plot above show what happens as the amount of gaming increases when setting y = 0, 0.25,0.5,0.75,1. 
Notice that the difference between the SVM curve and the curve with y = 1 is that the classifier for y = I 
is shifted according to our algorithm. 
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A Proof of Theorem 13.31 


Proof of Theorem \3.3\ Fix h eC. As in the proof of Theorem 2.3[ we begin by defining the set 
r(/) of X € X so that /(A(x)) = 1 when A is a best response to /. For every f eC, we have 

F(/) := {x : max{/(y) : y € F(x)} = 1} 

= {x : (3y eX,3be B)(f(y) = 1, b{x,y) < 2)} 

= {x : (3b € B)(bi{x) > min{b 2 (y) : f(y) = 1} - 2)} 

= : bi{x) > min{b 2 iy) : fiy) = 1 } - 2 }. 

Now we can again restrict our attention to nicely structured functions. For any f, let 


1 if {'^b e B){b 2 {y) ^ min{b 2 (z) : f{z) = 1}) 
-1 otherwise . 


f'iy) = 

Then, as in the proof of Theorem |2.3[ we have 

min{b 2 {y) : f{y) = 1} = min{b 2 {y) : f'{y) = 1} 

for all b e B. Indeed, 


(9) 


min{fo 2 (y) : f'iy) = 1} > min{fo 2 (z) : /(z) = 1} 

by definition of /', and 

min{fo 2 (y) -ffy) = 1} 

= mm{b2{y) : y e : hM > min{fo2(z) : f(z) = 1 }}} 

< min{fo 2 (y) : fiy) = 1} , 

using the fact that 

[y ■ fiy) = 1} c [w: b 2 {w) > min{l72(z) : /(z) = 1}} 

for all b e B. Thus, 

r(/) = Ufoggjx : bi{x)>min{b2{y) : /(y) = l}-2} 

= Ufoggjx : bi{x) > min{b2{y) : /'(y) = l}-2} = F(/') . 
Thus, as before, the payoff to Jury if she plays / is the same as if she plays /': 

lP{h{x) - max{/(y) : y e F(x)) 

= lP((T{f)Ah)^) 

= P((F(/0A/zr) 

= lP(h{x) = max{/'(y) : y e F(x)}) . 
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Thus, it suffices to consider classifiers / of the form Moving the quantifiers around, it 
suffices to consider classifiers of the form 


/ = min{l72[s{,] : b eB] 


( 10 ) 


for 

SB'=se {b2(X) U {oo}) . 
beB 

Here, sj, plays the role of min{b 2 (z) : f(z) = 1}, and S}, = oo means that f{z) = -1 for all z e X. 
For / as in (|10[|, we have 


r(min{F 2 [sfc] : beB}) = [J{x : Fi(x) >5^-2}, 


beB 


and a hest-possihle payoff to Jury is obtained hy finding the best thresholds s: 

OPT}i{V,c) = 1 - iiif |p(/2(x) ^ minjFJs;, - 2](x) : b € S})| 

=: 1 - inf {err(s)} . 

seSs 

In Algorithm]^ Jury returns 

/ =min{b2[sl] : b e B}, 
and as above the payoff to Jury from this / is 

P(/i(x) ^ Ci[s* - 2](x)) = 1 -err(s*) . 

As in the proof of Theorem|2.3[ to prove Theorem|3.3|it suffices to show that for all /i € C, 


err(s*) < inf{err(s) : s€Sg} + £ 


As before, we have 


eff(s*) = min{eff(s) : s € Sg}, 

so it suffices to establish that erf(s) is close to err(s), uniformly over s € Sg. 
Claim A.l. V^ith probability at least 1 - b, for all h eC and s € Sg, 

|err(s) - err(s)| ^ 4R^(C) + • 

In particular, if the hypotheses of the lemma are met, with probability at least 1-6, 

sup I |eff (s) - err(s)| :/i € C,s € Sgj < e/2. 


( 11 ) 

( 12 ) 


Proof. Again, this follows very similarly to the proof of Theorem 2.3 We need to bound the 
absolute value of the following difference: 


^^li^h{xj) ^min{bi[si, - 2]{xj) : b e B}] 


;=i 


{h{x) ^ minjFi [sj, - 2](x) : beB 
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for allheC and s € Sg. As before (via, say, Theorem 3.2 in |BBL05| ). we have for all h eC,s e S^, 


1 

m 


;=i 


E 




h{x)^minbi[ti,]{x] 

beB 


^ 2Rffi{A!) + 


/21n(2/h) 

V w 


where 


As before. 


X = Ih- minfoj[s^ - 2] : heC,s e 
[ beB 


RJAl)^2iRJC) + Rjn)). 


(14) 


where H = {minf,gg bi[si, - 2] : s € Sg}, so it remains to bound R^i'H). For fixed Xi,...,x^eX, 
we have 




a,minfoi[sfc-2](x;) : s€Sg 
I heB 

1=1 

/ m 

sup] ^^(^/•mini7i[s{,-2](x,) : s € ^{sy,^ : j € [m + 1]} 


E, 


1=1 


beB 


< 2 a/ 


ln((m+l)i®l* 


Thus, we have 


Rm{n )^2 


\B\ln{m + 1) 


m 


and, along with Equationthis finishes the claim. ■ 

As in the proof of Theorem |2.3[ Equation and Claim A. 1 finish the proof of Theorem 
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